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Executive Summary 


The Voluntary Framework of Accountability (VFA) is the principal institutional accountability and 
improvement framework for community colleges with measures defined to encompass the full 
breadth of the community college mission and the diversity of students’ goals and educational 
experiences. The American Association of Community Colleges (AACC) operates a web-based 
portal for community colleges to submit, analyze, and display their VFA data. In 2018, more than 
200 colleges participated in the VFA. This document is intended to provide a sample of results from 
the 2018 collection cycle. 


Key results from this report include: 


° The median student completion/persistence rate for the VFA participating colleges was 52.7%. 
In other words, half of the colleges had 52.7% or more of their students who started in the 
fall of 2011 earn a credential by 2017, transfer to another institution by fall of 2017, or be still 
enrolled in their sixth academic year. 


e Student outcomes and leading indicators of student outcomes varied by race and ethnicity. 
For most outcomes, White students had higher median rates than did Hispanic or African 
American students. Median success rates for African American students tended to be lower 
than White or Hispanic students with the exception of transfer rates, where African American 
students had the highest median transfer rate relative to White or Hispanic students. 


e Data suggest a relationship between the leading indicators collected in the VFA and 6-year 
outcomes collected in the VFA. Colleges with higher rates on the leading indicators tended 
to have higher rates on 6-year outcomes. 


The data provided in this report are intended to provide an overview of some of the data included 
in the VFA, and the opportunities it can provide community colleges in understanding student 
progress and outcomes. 


This report presents data from the colleges participating in the VFA. It is intended to provide an 
overview of the range of results on selected student progress and outcome metrics that the VFA 
collects. The data are not intended to provide a comprehensive picture of all community colleges, 
nor a complete representation of all the data collected through the VFA. Rather, it is intended to 
highlight some of the results for those colleges who have elected to participate in the VFA for the 
2018 participation year. 


What is the VFA? 


The VFA is the principal institutional accountability and improvement framework for community 
colleges with measures defined to encompass the full breadth of the community college mission 
and the diversity of students’ goals and educational experiences. 


Beginning in 2009, AACC, in collaboration with the Association of Community College 

Trustees and the College Board, began the development of the VFA. More than 60 community 
college leaders guided the 18-month development of the VFA resulting in the release of more 
appropriate measures than traditional accountability measures. The VFA gauges student progress 
and outcomes including pre-collegiate preparation (such as developmental education and Adult 
Basic Education), academic progress and momentum points, completion and transfer measures, 
and workforce outcomes for career and technical education. AACC subsequently developed 

a process for colleges to submit and display these data for internal institutional improvement 
purposes and for external accountability. 


Traditional accountability measures in higher education do not adequately measure the unique 
mission of community colleges. For example, measures may exclude part-time students or non- 
credit career and technical students who are a key part of community colleges’ mission. In light 
of the inadequacy of existing measures, the VFA provides community colleges with a significantly 
improved ability to assess their performance, identify areas for improvement, and demonstrate 
their commitment to their academic mission. 


The VFA is continually working to ensure that the measures collected effectively provide meaningful 
data for community colleges. For example, since the inception of the VFA, community colleges 
have been evolving the way developmental need is assessed, and how developmental coursework 
is delivered; therefore, the VFA refined the collection of developmental progress metrics. In recent 
years, many colleges have been participating in implementing structured or guided pathways in 
their institutions. AACC, along with several partner organizations, launched the AACC Pathways 
Project, which has been using leading indicator measures to track student progress. These 
measures have been incorporated into the VFA. 


One set of measures provided in the VFA are outcomes of a cohort of new students at the end 
of 6 academic years. All of the students in this cohort will fall into one of nine mutually exclusive 
categories: 


Student Outcomes at the End of 6 Years 


e Bachelor’s Degree: reporting college awarded bachelor’s degree 
e Associate Degree: reporting college awarded associate degree 
o Transfer after receipt of degree 
o No Transfer after receipt of degree 
° Certificate: reporting college awarded a formal award (certificate) 
o Transfer after receipt of award 
o No Transfer after receipt of award 


e Transfer, No Award: transferred to another post-secondary institution without earning a formal 
award at the reporting college 
e Still Enrolled: still enrolled during the sixth academic year 
e Left, No Award: not enrolled during the sixth academic year 
o Earned = 30 college-level credits at reporting institution 
o Earned < 30 college-level credits at reporting institution 


By collecting these nine separate outcomes, colleges are able to better understand a wide range of 
student outcomes, and better document outcomes of students. Unlike most measures of outcomes, 
the VFA allows colleges to document all transfers—not only transfers for students who did not earn 
a credential. For the purposes of this analysis, we will create metrics by combining these outcomes 
in a variety of ways. 


Six-Year Outcomes 


The VFA was developed to provide community colleges with better metrics, more appropriate to 
their missions and the students they serve. The 6-year outcomes provide colleges an opportunity 
to examine outcomes for students 6 years after entering their institution. Six years provides more 
opportunities for students who do not enroll full time or continuously to attain their goals, yet not 
so long that it becomes policy irrelevant. As noted above, the VFA provides a range of outcomes 
that colleges can examine to understand what happened to an incoming cohort of students 6 years 
later. 


For analytical purposes, several of these outcomes will be combined to give a sense of the range of 
outcomes for students at the colleges participating in the VFA. While creating a combined metric 
is illustrative, outcomes of students at individual colleges will be best understood by exploring the 
component parts of the summary metrics. The first metric reported here is completion/persistence 
for the incoming cohort. This metric is calculated by counting all students who either earned a 
credential at the reporting college, transferred to another college, or were still enrolled at the 
reporting college at the end of 6 years. Therefore, it consists of completion at the college, or 
persistence either at the college or by transfer. 


As shown in Figure 1, the median completion/persistence rate for VFA participating colleges for 
all students who began in the fall of 2011 (VFA Main Cohort’) was 52.7%. In other words, half of 
the colleges had completion/persistence rates above 52.7% for their main cohort of students. 
The figure also demonstrates that there was some variability across colleges on this measure. For 
example, the range between the bottom 10% (bottom decile) and top 10% (top decile) on the 
completion/persistence rate was 42.0% for the bottom decile and 61.5% for the top decile. 


'The VFA collects three cohort types. The VFA Main Cohort includes every student who entered the institution for the first time 
after completing highs school (either as a first time student or a transfer in student). 


Figure 17 


Distibution of Completion / Persistence Rates for VFA Colleges: Fall 2010 Cohorts 
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One challenge when measuring community college student outcomes is that not all students who 
enroll actually intend to earn a credential. The VFA collects a separate cohort of incoming students 
who are identified as the Credential Seeking Cohort. These are students whose behavior indicates 
that they are attending with the intention of earning a credential. Figure 1 shows that the median 
completion/persistence rate after 6 years for these students (65.6%) is higher than for all students 
who began in the fall of 2011 (Main Cohort). 


The VFA also collects data on one final cohort type, the First-time in College (FTIC) Cohort, which 
consists of students whose first enrollment in postsecondary education following high school is at 
the college. The median completion/persistence rate across the colleges for this cohort is slightly 
lower (51.1%) than the VFA Main Cohort (all students). 


The completion/persistence rate includes students who complete a credential, students who 
transfer without a credential, and students who are still enrolled. To better understand student 
outcomes after 6 years, the data are broken out into some of the component parts. The first data 
represented (see Figure 2) are the percent of students who earned a credential at the college by 
the end of 6 years (completion rate). The median VFA college completion rate for the VFA Main 


2See appendix 1 for a guide to interpreting the Box and Whisker Plots presented in this report. 


Cohort was 23.3%, with the range of the bottom decile of colleges being 15.8% and the top decile 
of colleges at 33.2%. This range of completion rates reflects the different policy environments for 
VFA colleges in which some states provide incentives to attain a credential prior to transfer while 
other states do not. When limiting the analysis to the Credential Seeking Cohort, there is a clear 
increase in the rate of completions across all colleges with the median rate at 37.2% and 10% of 
the colleges nearly at or above 50% completion rates. 


Figure 2 


Distribution of Completion Rates for VFA Colleges: Fall 2010 Cohorts 
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The VFA also captures the rate of students who transfer to another postsecondary institution. 
The data are reported in a way that allows colleges to differentiate students who transfer out 
after earning a credential, and those who transfer out without a credential (see Figure 3). For the 
VFA Main Cohort, the median transfer rate without a credential is one in four (25.0%), while the 
median transfer rate with a credential is nearly one in 10 students (9.9%), which translates into 

a median of 35.9% overall transfer rate. The median transfer rate without a credential is similar 
for the FTIC cohort (22.1%) and Credential Seeking Cohort (23.1%); however, the transfer with 
credential rate is higher for the Credential Seeking Cohort (16.2%). 


Figure 3 


Distribution of Transfer Hates for VFA colleges:2010 Main Cohort 
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Leading Indicators 


The outcomes of students after 6 years provides a rich understanding of the long-term outcomes 
of students who entered an institution; however, 6 years is a long time to wait to see the impact 
of institutional changes, and the extent that they are influencing student outcomes. Therefore, 
colleges need earlier indicators of student success that can be used to evaluate how well students 
are achieving their goals. The Community College Research Center (CCRC) has been one of the 
leading organizations looking into the value of leading or early indicators, and suggests that these 
can be productive in indicating longer term community college student success.* Leading indicators 
have been incorporated into the AACC Pathways initiative, and CCRC continues to explore the 
value of these measures. 


The VFA incorporated a variety of early or leading indicators of success in the original metric 
development, and augmented them in the most recent data collect cycle with measures used in 
the AACC Pathways initiative. This report will not present all of the measures, but will highlight a 
few of the measures. The VFA collects these measures for the 6-year cohort, as well as for more 


3See Jenkins, D. & Bailey, T. (2017) Early Momentum Metrics: Why They Matter for College Improvement. (CCRC Research Brief 
No. 65). New York, NY: Columbia University, Teachers College, Community College Research Center. httos://ccre.tc.columbia.edu/ 
publications/early-momentum-metrics-college-improvement.htm| 


recent cohorts of students. This report will limit the findings to the 2011, 6-year cohort to illustrate 
the relationship between the progress and early indicator measures and the 6-year outcomes. 


Figure 4 shows the distribution of VFA college outcomes for a variety of credit threshold measures. 
The median rate of students in the VFA Main Cohort who earned six or more credits in the first 
term was 47.9%, while the median rate of students in the VFA Main Cohort who earned 15 or more 
credits in the first year was 33.2%. Both of these rates are below the median 6-year completion/ 
persistence rate (52.7%) shown in Figure 1 for the VFA Main Cohort. Many students who attend 
community college do not attend full time, and measures need to be sensitive to this issue. The 
Reached Credit Threshold metric is defined to measure different levels of credits attained at the 
end of 2 years, based on whether students started full time or part time their first term. Using this 
metric, the median rate for the VFA Main Cohort was 35.0%.* These metrics provide colleges a set 
of indicators at the first term, the end of the first year, and the end of the second year to gauge how 
well students are progressing toward credential completion or transfer. 


Figure 4 


Distribution of Early Indicator (Credit Milestones) Rates for VFA Colleges: 2010 Main Cohort 
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3 Students who began their first term full time need to earn 42 credits by the end of 2 academic years, and students who began 
their first term part-time need to earn 24 credits by the end of 2 academic years to meet the Reached Credit Threshold definition. 


Other areas that CCRC has identified as important leading indicators are whether students complete 
college-level English and college-level math in the first year. Figure 5 provides data on the median 
rates of VFA colleges on these two metrics: 37.2% was the median rate of students completing a 
college-level English class in the first year, and 17.9% was the median rate for completing college- 
level math. 


Figure 5 


Distribution of Select Early Indicators for VFA colleges: 2010 Main Cohort 
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Finally, the VFA looks at another early persistence metric: fall-to-next-term retention rates. As shown 
in Figure 5, half of the VFA colleges had fall-to-next-term retention rates of 71.4% or higher; 90% 
of the colleges had fall-to-next-term retention rates above 59%. 


Achievement Gaps 


Community colleges serve diverse populations. The VFA collects disaggregated data across 
several student characteristics to help colleges better understand how well they are serving these 
diverse populations. Student characteristics colleges can use in the VFA are gender, racial/ethnic 
background, age, first-term enrollment status, level of developmental need at entrance, and 
whether or not the student received a Pell Grant during his or her enrollment at the college. 


This report will begin to explore one of those disaggregation: the racial/ethnic background of the 
students. For this report, we will examine the progress and outcome metrics of the three largest 
racial/ethnic groups reported in the VFA: White, Hispanic, and Black/African American students. 


The data reported for the 2017 collection cycle indicated differences in the median completion/ 
persistence rate for the VFA Main Cohort. The median rate was 54.7% for White students, 51.3% for 
Hispanic students, and 50.9% for African American students (see Figure 6). In other words, Hispanic 
and African American students were less likely than White students to have either graduated, 
transferred, or still be enrolled after 6 years. 


Figure 6 


Median Six Year Outcomes by Selected Race/Ethnicity for VFA Colleges: 2010 Main Cohort 
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When looking at the separate components of the 6-year rates, there are differences in the pattern 
among the different groups. The median rate for students earning a credential at the colleges was 
25.5% for White students, 20.1% for Hispanic students, and 12.3% for African American students. 
However, the transfer rate (with or without a credential) was 36.6% for White students, 33.0% for 
Hispanic students, and 38.4% for African American students. In other words, African American 
students were least likely to have earned a credential, but most likely to have transferred over the 
6-year period. 


To better understand these outcomes, the progress and early indicator measures can help colleges 
understand early differences that may lead to different 6-year outcomes and achievement gaps. 


The median fall-to-next-term retention rates, shown in Figure 7, were 72.4% for White students, 
72.7% for Hispanic students, and 67.9% for African American students, indicating African American 
students were less likely to return in the fall term than were Hispanic or White students. 


Figure 7 
Median Early Success Indicators For VFA Colleges by Selected Race /Ethnicity: 2010 Main 
Cohort 
80% 
70% 
60% 
50% 
40% 
30% 
20% 
: ; a 
: Ld 
Fall to Next Term Retention Complete College Level English Completed College Level Math 


White MAfricanAmerian MS Hispanic 


AACC's Pathways Project has developed a series of first-year metrics that are useful to indicate 
students’ path toward likely success. These indicators show that even in the first year there are 
significant gaps in outcomes across race/ethnicity. For example, the median rate for students passing 
an English course in their first year was 40.3% for White students, 36.2% for Hispanic students, and 
23.6% for African American students. Median rates for passing math in the first year were lower for 
all groups at 19.4% for White students, 14.9% for Hispanic students, and 7.1% for African American 
students. Early evidence from the Guided Pathways initiative suggests that passing both English and 
math in the first year are strongly related to positive student outcomes in the college’. 


Jenkins, D. and Bailey, T. (2017). Early Momentum Metrics: Why they Matter for College Improvement. New York, NY: Columbia 
University, Teachers College, Community College Research Center. 


Relationship Between Leading Indicators and Outcomes 


In order to get a better understanding of the relationship between the leading indicators and the 
6-year outcomes, a correlational analysis was conducted using the colleges as the unit of analysis. 


All colleges with sufficient data to calculate the AACC Pathways Key Performance Indicators (KPIs) 
and 6-year outcomes (N = 172) were included in the analysis. A simple Pearson correlation was 
conducted to ascertain the relation between leading indicators in a college (AACC Pathways KPIs) 
with the 6-year outcomes in the college. The analysis reveals that there is a significant relationship 
between many of the leading indicators and the 6-year outcomes. 


As shown in Table 1, colleges that had higher rates for students completing six or more credits in 
term 1, or 15 or more credits in year 1, also tended to have higher completion/persistence rates 
and credential completion rates for the same cohort at the end of year 6. Colleges with a higher 
percentage of students reaching the credit threshold by the end of year 2 were also more likely 
to have higher rates of completion/transfer and are much more likely to have higher credential 
completion rates. 


Table 1 


Correlation between VFA Colleges for Select Early Indicators and Select Six-Year Outcomes: 
2011 Main Cohort 


Persistence/Comple- 0.518 ** O32 O75 0.453 ** 0.437 ** 
tion Rate 

Credential Comple- 0.549 ** O629->* OT 0.378 °* O51" 
tion Rate 

Transfer Rate (with or 0.211 ** 0.193 * 0.207 ** 0.305 ** 0.142 


without credential) 


*p < .05; ** p < .01 


Early indicators were less related to transfer rates, with the exception of completing college English 
in the first year. Colleges that had higher rates of students completing college English in the 
first year were somewhat more likely to have higher transfer rates (with or without a credential). 
Completing college-level math was related to credential completion and completion/persistence 
rates. 


Appendix 1 


The chart below is designed to help interpret the Box and Whisker charts used in this report. 
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In this example, 90% of the colleges had rates above 10.4%, 75% had rates above 13.2%, half the 
colleges had rates above 17.9%, 25% of colleges had rates above 23.8%, and 10% of the colleges 
had rates above 29.1%, 


Conclusion 


The Voluntary Framework of Accountability provides colleges with a rich source of data for 
measuring and tracking student progress and outcomes. This report shows the importance of 
looking at a variety of measures to understand how well colleges are serving the educational goals 
of their students. In addition, it highlights the importance of disaggregating data to investigate 
and identify achievement gaps for different student populations. Finally, it provides additional 
support for the value of using the AACC Pathways KPIs as leading indicators for student outcomes. 


This report is a publication of the American Association of Community Colleges. 
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